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Summary of Review 

This review examines the reeently released Thomas P. Fordham Institute report, Education Olym- 
pics: The Games in Review. Published just after the eompletion of the 2008 Beijing Summer 
Olympics, Education Olympics strategically parallels the international competition by awarding 
gold, silver and bronze medals to top performing countries based on indicators including scores 
from international assessments in reading, mathematics, and science. The report contrasts Ameri- 
can students’ unimpressive performance on international assessments with the United States’ suc- 
cess in the Olympics. However, the report fails to substantiate its primary claim: that American 
students’ relatively low rankings on these tests will weaken the U.S. economy and jeopardize its 
future global standing. It also fails to substantiate secondary claims, set forth throughout in various 
sidebars. The report recognizes its numerous methodological weaknesses, but it nonetheless bases 
its conclusions primarily on findings produced by this flawed process. In addition, the research 
meant to bolster the report’s position is very limited. Ultimately, its conclusions lack a basis of 
argument or evidence, and its attempt to link test scores to the nation’s economic standing fails. 
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Review 



I. Introduction 

The current economic crisis provides a 
poignant frame for considering the claims 
contained in the Thomas P. Fordham Insti- 
tute’s Education Olympics: The Games in 
Review} This report, like many earlier ones, 
warns that the United States’ economy will 
suffer unless there are improvements in 
America’s mediocre showings on interna- 
tional assessments of student achievement. 

It is true that a statistically positive correla- 
tion can emerge in a comparison of different 
countries’ average test scores and their mac- 
roeconomic indicators. Indeed, the link 
between educational outcomes and the na- 
tion’s economic future has become conven- 
tional wisdom in popular political and poli- 
cymaking discourse. Yet, as will be dis- 
cussed later, the connection is far from set- 
tled, and the real-world relationship between 

a nation’s economy and its students’ scores 

-2 

is extremely attenuated. By way of illustra- 
tion, blame for the Dow-Jones Index’s pre- 
cipitous decline, the tanking of several ma- 
jor financial institutions, and the concomi- 
tant world economic crisis'^ cannot be laid at 
the feet of U.S. students’ often-middling 
performance on international assessments. If 
such troubling performances in education 
and the economy do share a common thread, 
that thread is the lack of carefully crafted 
and administered policies for governing 
many of our national systems, including 
those for economic and human develop- 
ment — not poor test scores. 

The Education Olympics was published just 
after the completion of the 2008 Beijing 
Summer Olympics, and it mirrors that inter- 
national competition with the award of gold, 
silver and bronze medals to top performers 



in various academic “events.” These compe- 
titions highlight the United States’ unim- 
pressive performance on international as- 
sessments of students’ reading, mathematics, 
and science. The report found the United 
States similarly wanting in international 
comparisons of upper secondary school 
graduation rates (i.e., high school in the 
United States) and the percentage of bache- 
lor’s degrees earned among college atten- 
dees. The only area in which U.S. students 
exceeded those of all other nations was in 
civic education. The report encourages read- 
ers to use a sensible approach when inter- 
preting the results, as it recognizes the me- 
thodological flaws of the report (i.e., sim- 
plistic ranking of nations, inconsistency with 
nation participation, and lack of statistical 
rigor in collapsing the results of the different 
international assessments). 

Calls for education reform, including those 
aimed at boosting students’ international 
standings, have always been situated in po- 
litical contexts. The launching of Sputnik 
led to an increased amount of attention in 
teaching U.S. students mathematics and sci- 
ence. The troubled economy of the early 
1980s inspired the writing of A Nation at 
Risk and its focus on higher standards and 
greater accountability.^ During the 1990s, 
concerns about inconsistent or indifferent 
efforts by U.S. students and educators again 
became politicized and resulted in the in- 
creasing use of high-stakes testing in public 
schools across the country. 

The Education Olympics report fits well into 
this tradition. The authors state that “The 
purpose of the Education Olympics is to 
contrast America’s tepid academic perform- 
ance with its athletic dominance” (page 2). 
However, the true purpose appears rather 
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different. The report strives to endorse a 
politieal vision that has dominated edueation 
reform for more than a deeade: that market- 
driven solutions, ineluding sehool ehoiee, 
vouehers, and inereased private-seetor par- 
tieipation in edueation, will improve sehools 
and lift student performanee. For instanee, 
the report’s sidebars, diseussed below, raise 
doubts about the use of edueational re- 
sourees in the U.S., while praising other 
eountries’ sehool ehoiee polieies and Can- 
ada’s laek of a federal role for edueation. 
The assertions made in the sidebars are 
wholly uneonneeted to the text’s presenta- 
tion of rankings and are not supported by 
them. 

II. The Report’s Findings and 
Conclusions 

The Education Olympics is based on previ- 
ously reported results of international as- 
sessments, and the findings reiterate those 
previous results that plaee U.S. students as 
middling performers in math, science, and 
reading. However the report seeks to convey 
these findings in a new way. To illustrate, in 
the overall mathematics literacy “event” 
(reported in Table 6), Chinese Taipei won 
the gold, Finland took silver, and Hong 
Kong and South Korea tied for bronze on 
the 2006 Programme for International Stu- 
dent Assessment (PISA). In contrast, the 
Education Olympics puts the US in 35^’’ 
place among a field of 57 competing na- 
tions. 

The Education Olympics constructed 58 
events and awarded a total of 190 medals. 
The top medal winners were Finland, with 
35, Hong Kong, with 33, and Singapore with 
16. The U.S. was credited with a single 
medal, a gold for achievement on the 
CrVED exam. 

The Education Olympics maintains that U.S. 



students’ academic achievement, in com- 
parison with those of rival nations, places 
the country at risk for losing its status as an 
educational leader and jeopardizes Amer- 
ica’s economic advantage. However, this 
conclusion is based on a report that goes to 
great lengths to point out that its methodol- 
ogy is not “a full-fledged scholarly analysis 
of international data” (page 7). 

III. The Report’s Rationale FOR ITS 
Findings and Conclusions 

The rationale for the report’s findings and its 
conclusion appears to rest on the presenta- 
tion of 28 tables of countries’ rankings on 
various student achievement measures. 
These tables were constructed from scores 
generated by various administrations of four 
international assessments of student 
achievement: the Trends in International 
Mathematics and Science Study (TIMSS, 
1999, 2003), the Progress in International 
Reading Literacy Study (PIRLS 2006), the 
Programme for International Student As- 
sessment (PISA 2003, 2006), and the Civic 
Education Study (CIVED 1999). Two addi- 
tional indicators included in determining 
medal winners were upper-secondary school 
(high school in the U.S.) completion rates 
and percentages of bachelor’s degrees 
awarded. 

The first table rank orders the countries by 
total number of medals. The next 15 tables 
note the gold, silver, and bronze medalist 
countries in various academic competitions. 
These tables present America’s overall rank- 
ing in the category along with the three 
medal-winning countries. 

The remaining twelve tables rank all partici- 
pating nations from highest to lowest in vari- 
ous competitions. These twelve tables indi- 
cate that U.S. students typically place in the 
middle of the international rankings of 
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achievement and attainment. This is the evi- 
denee the report provides for its initial asser- 
tion that in international eompetitions in edu- 
eation, the United States fares mueh worse 
than in international eompetitions in sports. 
However, the report never makes an explieit 
argument as to how its data support its een- 
tral eonelusion. That is, there is no elear link 
between these tables and the report’s eon- 
elusion that U.S. students’ performanees 
refleet a threat to U.S. eeonomie dominanee. 
The report alludes to research in this area by 
Hanushek, Jamison, Jamison, and Woess- 
mann.^ Its eonelusion is essentially offered 
as self-evident. 

IV. The Report’s Use of Research 
Literature 

To support their elaim that Ameriean stu- 
dents’ middling performance will undermine 
the nation’s eeonomie well being the Educa- 
tion Olympics report referenees one pieee of 
researeh: by Hanushek, et al., whieh was 
published in Education Next. That article, 
whieh is briefly diseussed in the report’s 
prefaee, seeks to equate national perform- 
anee on international math and seienee tests 
to a national level of cognitive skills avail- 
able to enter the workforee. As diseussed 
below, this equating is problematie. 

Following the presentation of Hanushek, et 
al.’s researeh, the Fordham report addresses 
the eontradietion that Ameriean students’ 
typieally ordinary performanee on interna- 
tional measures eoexists with the higher- 
than-average U.S. Gross Domestie Product 
growth rate. Drawing on Hanushek, et ah, 
this diserepaney is explained by the expan- 
sion of U.S. seeondary and higher edueation 
throughout mueh of the last eentury and by 
eeonomie advantages, ineluding weaker 
trade unions, lower tax rates, and fewer reg- 
ulatory restrictions. The report then under- 
seores an assertion made by Hanushek, et al. 



that, “we’re living on borrowed time.” It 
claims that while the United States has stood 
still, other countries have surpassed U.S. 
educational participation and attainment and 
have gained on our eeonomie edge. 

Other researeh literature is not diseussed, 
although the report’s prefaee mentions that 
researchers Gerald Bracey and Iris Rotberg 
have eritiqued international comparisons of 
student aehievement. One brief paragraph of 
the report’s preface takes issue with Wash- 
ington Post reporter Jay Mathews because, 
in a Wilson Quarterly artiele not eited in the 
Education Olympics, Mathews “seems to 
insinuate that ... test seores have no rela- 

o 

tionship to eeonomies.” 

There are, in faet, extensive bodies of re- 
seareh that respeetively eouple or deeouple 
edueational aehievement and eeonomie out- 
eomes. By way of illustration, in the former, 
there is work not only by Hanushek and his 
eolleagues, but by Riehard Mumane and 
Frank Levy,^ John Bishop, Robert Cos- 
trell,^^ and Ludger WoBmann and his col- 
leagues. In the latter, in addition to Bracey 

and Rotberg, there is work by Henry Levin, 

1 

Lawrenee Mishel and Riehard Rothstein, 
as well as Franeiseo Ramirez and his eol- 
leagues. 

Alongside researeh produeed by sueh econ- 
omists and sociologists, cognitive psycholo- 
gists and psychometricians have long de- 
bated the relationship between test scores 
and eeonomie suceess. Helmuth Nyborg and 
Arthur Jensen, in addition to Charles Mur- 
ray and Riehard Hermstein,'^ have under- 
seored the relationship between tests seores 
and worker ineome. In eontrast, Stephen 
Ceei in a reanalysis of data from high-IQ 
youngsters originally assembled by Lewis 
Terman,^^ found that the eeonomie lives of 
those with high IQs were greatly influeneed 
by whether they came of age during or after 
the Great Depression. Susan Barnett and 
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Stephen Ceei’s views on test seores and 
eeonomie sueeess have deseribed the impor- 
tanee of the dimensions of “knowledge do- 
main” and “temporal eontext” to students’ 
futures. They define the knowledge domain 
as the knowledge base to whieh the skill or 
knowledge is to be applied, while the tempo- 
ral eontext is the elapsed time between the 
training and testing phase. They have 
stressed that “these dimensions have praeti- 
eal relevanee to transfer from sehool elasses 
to future life, as does physieal eontext.”'^ 
That is, they have relevanee to the ability to 
extend what has been learned in one eontext 
to new eontexts. 

Other eognitive psyehologists, ineluding 
Howard Gardner and Robert Sternberg, have 
argued that standardized tests, whether 
aehievement or IQ, refleet limited samples 
of both human eognition and of the human 
behaviors needed for workplaee perform- 
anee. To illustrate, these tests provide no 
opportunity to examine test takers’ eapaeity 
to prioritize, eooperate, persist in the faee of 
serious challenges, or generate and make use 
of critical feedback. Hence, there is a com- 
plex and attenuated relationship among test 
results, worker success, and overall macro- 
economic performance. 

V. Review oe the Report’s 
Methods 

Education Olympics takes existing findings 
from international assessments of educa- 
tional achievement and attainment and then 
orders the reported scores from highest to 
lowest. In Chapter 1 of the report, “Tally of 
Medal Winners,” scores were listed from 
highest to lowest, without indication of 
whether ranked scores reflected any statisti- 
cally significant differences. Test scores 
only provide approximations of true knowl- 
edge or ability, and careful analyses always 
take into account whether differences are 



significantly different. The new Fordham 
report does not, however, do this. The scores 
were ordered simply on the basis of higher 
or lower numbers. Then, to parallel the 
Olympic Games, gold, silver or bronze med- 
als were given to the three highest-scoring 
countries. However, without knowing 
whether the scores were meaningfully dif- 
ferent or instead differed just by chance, it 
makes little sense to distinguish among 
countries. 

The Education Olympics awarded a total of 
190 medals based on 58 events that were 
developed from existing international rank- 
ings from the various assessments, as well as 
upper secondary school graduation rates and 
bachelors degrees awarded. Each country 
was given a final ranking based on the num- 
ber of medals earned. With its one medal, 
the U.S. was ranked 20th among the 77 par- 
ticipating nations. 

However, as the report itself notes, the over- 
all number of medals might in part reflect 
the varying number of international assess- 
ments that a country participated in. This 
undermines the value of this method of 
comparing countries because a small num- 
ber of medals does not necessarily reflect 
actual student achievement in a country but 
instead reflects, to varying extents, partici- 
pation. 

In Chapter 2 of Education Olympics “Inter- 
national Assessments and Results,” the au- 
thors note that “Unlike the medal events in 
Chapter 1, ... we report international data . . . 
consistent with how it’s reported by the Na- 
tional Center for Education Statistics 
(NCES) and the Organisation for Economic 
Co-operation and Development (OECD)” 
(page 20). That is, the rankings now take 
into account whether scores were statisti- 
cally distinguishable from one another. The 
participating countries are rank ordered into 
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banded groups of high, middle, and low, 
with the United States shown to be in the 
middle of nearly all the rankings. 

However, this approach, too, is problematic. 
The rankings are presented in simplified and 
misleading ways. To illustrate, the averag- 
ing, selection, and presentation of data and 
findings in the Education Olympics’ ob- 
scures, rather than illuminates, the consider- 
able variability within and across U.S. stu- 
dents’ performances. For example, in Chap- 
ter 1, U.S. 4* grade math scores on the 2003 
TIMSS are included in the analysis but, for 
reasons never explained, those students’ 
science scores were not separately discussed 
or ranked. This is noteworthy because the 
math scores were right in the middle of the 
international pack overall, and three G8 
countries attained statistically higher 
scores. In contrast, U.S. 4* grade science 
scores on the 2003 TIMSS turn out to be 
third highest, with only one G8 country 
scoring better. There is clearly more to 
these assessment results than the preface’s 
twice-made claim that the U.S. generates 
“mediocre international results.” 

VI: Review oe the Validity oe the 

Findings and Conclusions 

The valid use of test scores for any given 
purpose rests on the establishment of an 
integrated set of arguments that are sup- 
ported by theory and evidence. Education 
Olympics attempts to use the scores from 
international assessments of education to 
conclude that U.S. students’ middling scores 
threaten the nation’s economic well-being 
and its international economic standing. 
However, the content of the report provides 
no basis of argument or evidence to support 
that claim. It does not consider, and thereby 
cannot counter, research that suggests only a 
tenuous relationship between national eco- 

2S 

nomic strength and school performance. 



Rather than examining the possible relation- 
ship between education and economic out- 
comes, the report simply takes it as a given 
that the relationship is straightforward and 
strong. 

Most notably, the report fails to establish a 
connection between any economic indicators 
and the 26 tables of ranked test results it 
presents as findings. In essence, arguments, 
theory, and evidence are entirely absent 
from the section of Education Olympics 
attempting to link the rankings to economic 
issues. Without any such support, the re- 
port’s conclusions that student achievement 
influences national economic well-being 
cannot and should not be considered valid. 

Instead of developing careful, well- 
reasoned, and clearly supported arguments 
and conclusions, the authors have relied on 
ten sidebars to make a number of claims 
about education and economic performance. 
These sidebar assertions tend to overlap with 
Fordham Institute policy initiatives for U.S. 
schooling, including the following: setting 
classroom size standards for optimizing in- 
struction, increasing opportunities for school 
choice, and setting optimal per-pupil expen- 
ditures. For instance, a sidebar entitled 
“Healthy Choice” describes how students in 
top-performing countries were able to enroll 
in schools based on school performance. 

Top performing nations, overall, offer 
at least some degree of school choice. 
For instance, according to PISA 2006 
data, roughly 90% of students in 
Hong Kong and Australia are en- 
rolled in schools where principals re- 
port that two or more schools are 
competing for students in the same 
area. . . . Compare that to the roughly 
60% of Americans experiencing the 
same. ... All our other top medal 
winners are either equal to or over the 
OECD average of 60% (page 10). 
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Although the report does not explieitly state 
that sehool ehoice is correlated with higher 
performance on international assessments, 
the implication from the report is clear: 
school choice should he supported if the 
United States is to compete with countries 
that have high levels of performance on the 
PISA in science. Yet the sidehar’s content is 
driven by ideology rather than reasoned ar- 
gument — which is evident from the fact that 
Finland, which comes out highest in the 
report’s rankings, is also noted in the same 
sidebar to have the lowest rate of school 
choice. 

VII. Usefulness of the Report for 
Guidance of Policy and 
Practice 

The Education Olympics provides a limited 
view of U.S. educational performance based 
on international assessments. Although in- 
ternational assessments of education can 
provide useful information for policymakers 
and curriculum developers, they are often 
used without adequate consideration of the 
varied contexts in which these results are 
obtained. The U.S. education system differs 
considerably from that of many other na- 
tions. It has no central ministry of education, 
no national curriculum, and an extremely 
heterogeneous school population. Within its 
50 states and 13,500 school districts, there 
are vast differences in educational resources 
and challenges. Throughout much of the 
United States, there are increasing percent- 
ages of non-English speaking students^^ and 
students of color, even as the teaching 
force remains predominately white and fe- 
male.^^ Schools are markedly segregated 
both by race and class, with high-minority 
and high-poverty schools commonly having 
the least experienced and least well-trained 
teachers. Buried beneath the nation’s aver- 
age scores are school and district perform- 



ances that are all over the map. In this case, it 
makes little sense to attempt to build policy 
based upon the score averages, rankings — or 
anything else — provided in this report. 

In fact, the main body of Education Olym- 
pics does not attempt to provide clear guid- 
ance for policy or practice on the basis of its 
findings and conclusions. Instead, the report 
makes brief allusions to policy issues in si- 
debars, where the report uses anecdotes in- 
stead of real analysis to draw conclusions 
about what drives student performance. In 
addition to the school choice sidebar noted 
above, there are sidebars on per-pupil spend- 
ing (U.S. spending is noted as second high- 
est, suggesting that spending is not impor- 
tant), class size (many higher scoring na- 
tions have bigger classes), and the role of 
the federal government (highlighting “Cana- 
dian independence” from a federal role). 
The sidebars entail no research-based analy- 
ses of the needs of students, teachers, par- 
ents, communities, or other key stakeholders 
in public education. These indirect policy 
statements are also not built on any analysis 
of how prior reforms succeeded or failed to 
meet such needs. For example, school 
choice, which the report and the Fordham 
Institute clearly embrace, has already been 
incorporated into national policy, via the No 
Child Feft Behind Act (NCFB) and other 
policies. NCFB allows students in underper- 
forming schools to transfer to schools that 
have made adequate yearly progress. Yet, 
only a small percentage of eligible students 
have actually chosen to attend a different 
school. Making this and other forms of 
school choice available in the United States 
has also not changed the trajectories on the 
National Assessment of Educational Pro- 
gress (NAEP) or any other national school 
indicator. 

Genuinely improving educational perform- 
ance — that is, improving students’ ability to 
think, solve problems, and to transfer 
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knowledge and skills to new problems — is 
intrinsically worth pursuing, even if it can- 
not be linked to national economic health. 
However, such improvements are unlikely to 
happen without well-formed policies. Such 
policies must be based on thoughtful and 
thorough analyses of real-world needs. 



events, and contexts, not on ideology. The 
Education Olympics report, driven by prede- 
termined positions and lacking any rigorous 
demonstration of argument, theory, evidence 
or methods, provides no basis for generating 
constructive policy for improving our na- 
tion’s educational performance. 
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